Adaptive resonance architectures are neural networks that self-organize stable recognition categories in real time in response to arbitrary sequences of input patterns. The basic principles of adaptive resonance theory (ART) were introduced in Grossberg, "Adaptive pattern classification and universal recoding, II: Feedback, expectation, olfaction, and illusions." Biological Cybernetics, 23 (1976) 187-202. A class of adaptive resonance architectures has since been characterized as a system of ordinary differential equations by Carpenter and Grossberg, "Category learning and adaptive pattern recognition: A neural network model," Proceedings of the Third Army Conference on Applied Mathematics and Computing, ARO Report 86-1 (1985) 37-56, and "A massively parallel architecture for a self-organizing neural pattern recognition machine." Computer Vision, Graphics, and Image Processing, 37 (1987) 54-115. One implementation of an ART system is presented in U.S. application Ser. No. PCT/US86/02553, filed Nov. 26, 1986 by Carpenter and Grossberg for "Pattern Recognition System".
As shown in FIG. 1, ART networks encode new input patterns received at 20, in part, by changing the weights, or long term memory (LTM) traces, of a bottom-up adaptive filter 22. This filter is contained in pathways leading from a feature representation field (F.sub.1) to a category representation field (F.sub.2) of short term memory. Generally, the short term memory (STM) fields hold new patterns relative to each input pattern. The long term memory (LTM), on the other hand, defines patterns learned from some number of input patterns, that is, over a relatively longer period of time. This bottom-up filtering property is shared by many other models of adaptive pattern recognition and associative learning. In an ART network, however, it is a second, top-down adaptive filter 24 that leads to the crucial property of code self-stabilization. The top-down filtered inputs to F.sub.1 form a template pattern and enable the network to carry out attentional priming, pattern matching, and self-adjusting parallel search.
The fields F.sub.1 and F.sub.2, as well as the bottom-up and top-down adaptive filters, are contained within the ART's attentional subsystem. An auxiliary orienting subsystem 26 becomes active when a bottom-up input to F.sub.1 fails to match the learned top-down template from filter 24 corresponding to the active category representation at F.sub.2. In this case, the orienting subsystem rapidly resets the active category representation. This reset automatically induces the attentional subsystem to proceed with a parallel search. Alternative categories are tested until either an adequate match is found or a new category is established. The search remains efficient because the search strategy through filter 22 is adaptively updated throughout the learning process. The search proceeds rapidly relative to the learning rate. Thus significant changes in the bottom-up and top-down adaptive filters occur only when a search ends and a matched F.sub.1 pattern resonates within the system. The system carries out a search during many initial input trials. Thereafter, however, the search mechanism is automatically disengaged, with each input having direct access to its category representation.
In principle, any new input could create a new category at any time: plasticity, or the potential for change in the LTM, remains intact indefinitely. If at any time, for example, a new input were added to the previously learned set, the system would search the established categories. If an adequate match were found, the LTM category representation would be refined, if necessary, to incorporate the new pattern. If no match were found, a new category would be formed, with previously uncommitted LTM traces encoding the STM pattern established by the input. Nevertheless,the code does tend to stabilize as the category structure becomes increasingly complex, since then each new pattern becomes increasingly likely to fit into an established category.
The criterion for an adequate match between an input pattern and a chosen category template is adjustable. The matching criterion is determined by a vigilance parameter that controls activation of the orienting subsystem 26. All other things being equal, higher vigilance imposes a stricter matching criterion, which in turn partitions the input set into finer categories. Lower vigilance tolerates greater top-down/bottom-up mismatches at F.sub.1, leading in turn to coarser categories. In addition, at every vigilance level, the matching criterion is self-scaling: a small mismatch may be tolerated if the input pattern is complex, while the same featural mismatch would trigger reset if the input represented only a few features.
The orienting subsystem is one of the means by which an ART network carries out active regulation of the learning process. Attentional gain control 28 and 30 at F.sub.1 and F.sub.2 also contributes to this active regulation. Gain control acts to adjust overall sensitivity to patterned inputs and to coordinate the separate, asynchronous functions of the ART systems.